Dimensional Geometry , Curse of Dimensionality , Dimension Reduction

High-dimensional vectors are ubiquitous in algorithms and this lecture seeks to introduce some common properties of these vectors. We encounter the so-called curse of dimensionality which refers to the fact that algorithms are simply harder to design in high dimensions and often have a running time exponential in the dimension. We also show that it is possible to reduce the dimension of a dataset sometimes —and for some purposes. Notation: For a vector x ∈ <n its `2-norm is |x|2 = ( ∑ i x 2 i ) 1/2 and the `1-norm is |x|1 = ∑ i |xi|. For any two vectors x, y their Euclidean distance refers to |x− y|2 and Manhattan distance refers to |x− y|1. We start with some useful generalizations of geometric objects to higher dimensional geometry: